1. Field of the Invention
The present invention relates generally to an electronic imaging system and method, particularly, to an improved imaging system and method for customizing the images to the viewer""s specifications, and, more particularly, to an imaging system and method allowing the viewer, via a backchannel, to adjust the spatial and temporal resolution and quantization parameters of an image.
2. Background and Objects of the Present Invention
With the rise of the consumer electronics industry over the past few decades, a variety of electronic imaging systems of increasing complexity have emerged, e.g., video recorders, camcorders and the like. Additionally, video teleconferencing communications are becoming increasingly important as our society becomes increasingly and interactively interconnected.
As is understood in this art, video, i.e., moving, images undergo encoding to reduce the amount of information needed to represent a given image. Encoding affects both the spatial resolution, i.e., the detail within a particular image frame, and temporal resolution, i.e., the number of such image frames per second. These parameters are typically fixed within a conventional video system, such as the one shown in FIG. 1 of the Drawings and generally referred to herein as numeral 10. The video system 10 in the figure includes a sending device 12 which receives signals from a camera 14. It should be understood that various portions of camera 14 which are not related to the present invention, for example, the diaphragm, shutter and the like, are not illustrated. Accordingly, as is understood in this art, the optical image before the camera 14, such as the individual depicted, is received by a camera lens 16 and converted into an analog video signal, e.g., by a conventional charge coupled device. It should be understood that camera 14 may be a digital camera forwarding digital data to a subsampler device 18 within the sending device 12. If camera 14 is not digital, however, and analog-to-digital conversion is required, then device 18 may also function as an A/D converter, as is understood in the art. The subsampler 18 determines pixel values representing the captured video image at a particular spatial resolution, i.e., pixels per line and lines per image, and temporal resolution, i.e., images per second. Another parameter related to both spatial and temporal resolution is quantization, i.e., a measure of the amount of distortion present in the video signal, as will be discussed in more detail hereinafter.
An encoder 20 encodes the aforedescribed digital image data into a video signal stream which flows into a buffer 22. As is understood in the art and discussed further herein, the rate of the flow of information from the encoder 20 into buffer 22 varies in accordance with the degree of encoding. Additionally, the video signal stream typically includes compressed signals, in which image information has been condensed or compressed by the encoder 20 to facilitate transmission or storage. One set of formats using such compression technologies are those specified by the Moving Picture Experts Group (MPEG), a standard in accord with the International Organization for Standardization/International Electro-technical Commission (ISO/IEC). Other compression technologies are the H.261, H.262 and H.263 standards of the International Telecommunications Union, Teleconferencing Section (ITU-T) for use in video teleconferencing, for example.
In conjunction with these image data formatting standards and techniques, by which the encoder 20 provides a syntax for the subsequent bitstream, the encoder 20 also employs compression algorithms, such as Discrete Cosine Transforms (DCT), Huffman coding and other mechanisms, whereby the amount of data needed to represent the image is drastically reduced while substantially retaining image integrity. As is well understood by those skilled in the art, these and other techniques eliminate or reduce the transmission of frame-to-frame redundancies and other information which are unnecessary or repetitive, and exploit various physiological and psychological aspects of human perception to present a coherent image to the viewer""s eye.
With further reference to FIG. 1, the subsampler 18, encoder 20 and buffer 22 are controlled by a control unit 24, which also controls other functions of the imaging system 10. For example, control unit 24 controls the sequencing of the afore-described operations, i.e., image pickup by camera 14 through a connection thereto (not shown), pixel conversion in subsampler 18, compression in encoder 20, recording the encoded images on a magnetic or electronic recording medium (not shown), and other operations. Control unit 24 supplies encoder 20 with a plurality of operating parameters to govern the aforementioned transformation of pixel data into a corresponding compressed bitstream. As discussed, control unit 24 also governs the variable bit rate of the information flow into buffer 22 to maintain a particular data level and avoid both overflow and underflow therein.
As is understood in this art, the primary purpose of buffer 22 is to regulate the flow of data from the encoder 20 and forward that data at a fixed rate across a transmission channel 26 to a receiver device 28, particularly, to another buffer 30 therein, which like buffer 22 acts as a reservoir storing the data and regulating its use. It should, of course, be understood that channel 26 may transfer data at a variable rate, e.g., a variable rate service of an Asynchronous Transfer Mode (ATM) network. Nonetheless, the variable flow rate of data from encoder 20 does not generally agree with that of channel 26, fixed or variable.
Buffer 30 forwards the received image data, at a fixed or variable rate as needed, to a decoder 32. Similarly to the encoding process, the decoder 32 reverses the aforedescribed compression algorithms to expand the image pursuant to the aforementioned operating parameters. In other words, the decoder 32 decompresses the compressed information in the bit stream and reconstitutes the image pursuant to the relevant image format used, e.g., the ITU-R/601 Digital Studio Standard, and the operating parameters. The reconstituted image is then placed within an image storage device 34, the contents of which may be continuously displayed on a video display 36, the circuitry of which is understood in the art.
As discussed, the aforedescribed compression technologies employ various techniques to condense the image information. The decoder 32 is configured to interpret the format and operating parameters by which the image information was encoded by encoder 20. As is understood in the art, much of the decoding process performed within the decoder 32 may be called xe2x80x9cnormativexe2x80x9d, i.e., fixed by the particular standard used, e.g., MPEG. Consequently, the decoder 32 readily recognizes these normative parts of a signal from encoder 20, i.e., how to interpret the transmitted bits in the bit stream.
In conventional apparatus employing the above technology, the aforementioned operating parameters are fixed within the video system 10. Usually, encoder 20 utilizes fixed spatial and temporal resolution values, which comports well with the requirements of buffer 22, guaranteeing a fixed-rate bitstream across transmission channel 26. Nonetheless, buffer 22 in an effort to maintain the transmission rate required by the channel 26 adjusts the quantization or distortion of the pertinent images. Quantization then becomes a function of the fullness of buffer 22, which, in turn, is a function of the complexity of the subject video images, i.e., how bit-consuming the images are during compression. Some encoders 20 have fixed spatial resolution only and the buffer 22 adjusts quantization and temporal resolution to maintain the constant bit-rate. The balance between quantization and temporal resolution is governed by a buffer regulation algorithm, as is understood in the art.
One problem with the above configuration, however, is that the aforementioned operating parameters may be unsuitable in certain circumstances, and the buffer regulation algorithm or other resolution balancing scheme may require adjustment to suit the needs of the human viewer who may have a different spatial/temporal resolution and distortion balance in mind. For example, some video applications may require higher temporal resolution at the cost of coarse quantization, e.g., video communication between deaf people (sign language) who prefer high temporal resolution. Additionally, surveillance applications normally require higher spatial resolution and fine quantization at the cost of temporal resolution.
Further, with the growing rise of consumer use and proficiency in electronic imaging systems, sophisticated videographers want increasing control over the operating parameters and may make fine adjustments to the balance between spatial and temporal resolution and quantization for a multitude of other applications, fine tuning these parameters for objective or subjective effect.
With these operating parameters fixed, however, videographers or any other user of video apparatus having these immutable characteristics cannot make any adjustments to the apparatus and encoder 20 operates without any feedback from the viewer.
Accordingly, it is a first object of the present invention to provide the viewer with a means to adjust the aforementioned spatial and temporal resolutions and quantization variables to suit their individual needs.
It is more particular object of the present invention to provide a means of feedback from the viewer to the encoder, enabling the viewer to have increased flexibility over the aforementioned operating parameters.
The present invention is directed to an improvement in an imaging system and method for providing increased viewer control over the operating parameters of the imaging system. Viewer control is facilitated by including a backchannel within the imaging system, enabling the viewer to adjust the operating parameters, e.g., spatial and temporal resolution, and quantization, of the video transmission.
A more complete appreciation of the present invention and the scope thereof can be obtained from the accompanying drawings which are briefly summarized below, the following detailed description of the presently-preferred embodiments of the invention, and the appended claims.